1. Field of the Invention
This invention pertains in general to machine learning, and in particular to a Bayesian model of linear regression.
2. Description of the Related Art
Linear regression is a simple yet powerful approach to learning from data. The simplest algorithm for regression is ordinary least squares (OLS), which yields a hyperplane that minimizes the squared error between predictions and labels on a training set. In high dimensions with few examples, ordinary least squares can overfit badly, and needs to be regularized to produce sensible solutions. Additionally, it is often desirable to work in a Bayesian setting in order to produce confidence estimates and incorporate prior knowledge in our predictions.
In general, the Bayesian machine learning community frequently makes use of analytic Gaussian integrals, approximations such as using maximum likelihood to approximate a posterior with its maximum value, and Markov Chain Monte Carlo sampling, but does not make much use of modern tools for direct numerical low-dimensional integrations.